(1) Field of the Invention
This invention is related to color image sensor, particularly to color integrated image processing for a focal plane.
(2) Brief Description of Related Art
CMOS integrated circuits technology readily allows the incorporation of photodetector arrays and image processing circuits on the same silicon die. This has led to the recent proliferation in cheap and compact digital cameras, system-on-a-chip video processors, and many other cutting-edge commercial and research imaging products. The concept of using CMOS technology for combining sensing and processing was not spear-headed by the imaging community. It actually emerged in mid 80's from the neuromorphic engineering community, developed by Carver Mead and collaborators [1]. Mead's motivation was to mimic the information processing capabilities of biological organisms; biology tends to optimize information extraction by introducing processing at the sensing epithelium. This approach to sensory information processing, which was later captured with terms such as “sensory processing” and “computational sensors,” produced a myriad of vision chips, whose functionalities include edge detection, motion detection, stereopsis and many others examples can be found in references [2].
The preponderance of the work on neuromorphic vision has focused on spatiotemporal processing on the intensity of light (gray scale images) because the intensity can be readily transformed into a voltage or current using basic integrated circuits components: photodiodes, photogates, and phototransistors. These devices are easily implemented in CMOS technologies using no additional lithography layers. On the other hand, color image processing has been limited primarily to the commercial camera arena because three additional masks are required to implement red (R), green (G) and blue (B) filters. The additional masks make fabrication of color sensitive photodetection arrays expensive and, therefore, not readily available to researchers. Nonetheless, a large part of human visual perception is based on color information processing. Consequently, neuromorphic vision systems should not ignore this obviously important cue for scene analysis and understanding.
There has been a limited amount of previous work on neuromorphic color processing. The vast majority of color processing literature addresses standard digital image processing techniques. That is, they consist of a camera that is connected to a frame-grabber that contains an analog-to-digital converter (ADC). The ADC interfaces with a digital computer, where software algorithms are executed. Of the few biologically inspired hardware papers, there are clearly two approaches. The first approach uses separate imaging chips and processing chips [3], while the second approach integrates a handful of photodetectors and analog processing circuitry [4]. In the former example, standard cameras are connected directly to analog VLSI chips that demultiplex the video stream and store the pixel values as voltages on arrays of capacitors. Arrays as large as 50×50 pixels have been realized to implement various algorithms for color constancy [3]. As can be expected, the system is large and clumsy, but real-time performance is possible. The second set of chips investigate a particular biologically inspired problem, such as RGB (red,green, blue color)-to-HSI (Hue, Saturation and Intensity) conversion using biologically plausible color opponents and HSI-based image segmentation, using a very small number of photodetectors and integrated analog VLSI circuits [4]. Clearly, the goal of the latter is to demonstrate a concept and not to develop a practical system for useful image sizes.